# Text pre-training

Hebrew-Gemma-11B-V2 is an open-source Hebrew/English pre-trained generative text large language model with 11 billion parameters, based on Google's Gemma-7B architecture.

This is a distilled Portuguese BERT model derived from BERTimbau, retaining 99% of the original model's accuracy.